1. Field of the Invention
The present invention relates to an image processing system for processing digital image data and its image smoothing method, and more particularly, to an image processing system and its smoothing method characterized by the means employed for correcting an input image recorded in a color fog state and in a backlight state.
2. Description of the Related Art
When processing digital image data by use of a computer-based image processing system, various forms of smoothing are performed on the image data as necessary. Hereafter, color fog and backlight correction by use of a conventional image processing systems is described.
First, color fog correction is described. In the color fog correcting method used by conventional image processing systems, the tone curve of the RGB is interactively modified; the operator corrects the colors of an image while confirming the results of his operation.
This conventional method focusing on smoothing the tone curve of the RGB interactively, however, requires too many adjustment parameters for the operator to perform smoothing intuitively, thereby giving rise to a drawback of longer operation time. Furthermore, this method requires experience and skill so that those having no or little experience are unable to perform smoothing as desired.
This type of conventional image processing system for smoothing color is disclosed, for example, in Japanese Patent Laid-Open Publication No. 2-94893, xe2x80x9c*Image Processing Systemxe2x80x9d. The same publication describes a system which allows correction depending on the spectral properties of different types of light source.
The conventional image processing system disclosed in the same publication is described with reference to FIG. 14. In FIG. 14, a spectral properties storing means 1402 stores the amounts of correction to be made for three color components, i.e., red, green, and blue, depending on the spectral properties of different types of light source. A correction amount retrieving means 1403 retrieves the amount of correction for a specified light source from the spectral properties storing means 1402 and inputs it into a color correcting means 1401. The data stored in an input image buffer 1400 is subjected to color correction by the color correcting means 1401 and output to an output image buffer 1404. Assuming that the input pixel values are R, G, and B, and that the output pixel values are Rxe2x80x2, Gxe2x80x2, and Bxe2x80x2, color correction is performed using the following expression (1).
Rxe2x80x2=R/r0, Gxe2x80x2=G/g0, Bxe2x80x2=B/b0xe2x80x83xe2x80x83(1)
This correcting method multiplies the RGB value by the constant term.
In the correcting method that simply multiplies the RGB values by the constant term, however, it is impossible to smooth all the colors of respective light sources completely, resulting in the inability to achieve the best possible color through correction. In addition, a color fog state that has been caused by such factors as film, lens, and exposure characteristics, along with color variances among different light sources, is impossible to correct solely based on the properties of different light sources, making this method inadequate for obtaining good correction results.
Next, backlight correction is described. Conventional backlight correcting systems include an automatic exposure control device, for example, a video camera. This type of system controls the diaphragm so as to maintain the level of output signals constant. Means of controlling the diaphragm include the mean value method that obtains the mean value of brightness of an entire screen, the peak value method that detects the maximum value of brightness of a screen, and the method that combines both. A diaphragm control unit, typically located behind a lens system, controls the quantity of incident light to make the screen brighter or darker. This type of conventional backlight correcting system is disclosed, for example, in Japanese Patent Laid-Open Publication No. 6-46325, xe2x80x9c*Automatic Exposure Control Systemxe2x80x9d. The same publication describes a system that estimates the intensity of backlight based on the brightness data for a screen and determines how much the diaphragm should be controlled using the value thus obtained.
The conventional backlight correction device disclosed in the same publication is described with reference to FIG. 15. In FIG. 15, a subject image is formed on an image pickup device 1503 via a lens 1501 and a diaphragm 1502, converted into an electric signal, and output through a signal processing circuit 1505 for performing the xcex3 processing and other tasks. At this time, the diaphragm control is performed by use of a signal from the image pickup device 1503 as follows.
A full screen mean value detecting unit 1507 detects the mean brightness of the full screen. Subsequently, the diaphragm control unit 1506 determines the diaphragm aperture so as to achieve a target brightness. A smaller region mean value detecting unit 1508 divides a screen into smaller regions and calculates the mean value of each block. A low brightness mean value calculating unit 1509 arranges in the order of brightness the mean values of brightness of respective small regions calculated by the smaller region mean value detecting unit 1508, and selects a predetermined number of small regions sequentially from the small region of the lowest brightness, and calculates the mean brightness of the regions thus selected. A backlight degree calculating unit 1511 calculates the degree of backlight from the mean brightness of the several selected regions calculated by the low brightness mean value calculating unit 1509. A target brightness calculating unit 1510 modifies the target aperture so as to enlarge the diaphragm aperture when the degree of the backlight calculated by the backlight degree calculating unit 1511 becomes larger, and modifies the target aperture so as to reduce the diaphragm aperture when the degree of the backlight becomes smaller.
In conventional backlight correcting systems, backlight is corrected by adjusting the diaphragm aperture so as to enter the greater quantity of light or the smaller quantity of light. This aims to control the device, a video camera or the like, directly. The similar correcting method cannot be used for the data once quantized and stored as a digital image.
Considering diaphragm control as image processing, xcex3 correction conversion as illustrated in FIG. 16 is the closest conversion to backlight correction through the above-mentioned modification of the diaphragm aperture. When diaphragm is not changed, the input value of brightness is the same as the output value of brightness, as illustrated in the conversion characteristic 1603. An enlarged diaphragm aperture has characteristic as illustrated in the conversion characteristic 1601 for a large aperture and a reduced diaphragm aperture has the characteristic as illustrated in the conversion characteristic 1602 for a small aperture.
Since the range of partial light quantities in the scenery of the natural environment is being recorded by a video camera, the brightness value in the shadow region (the darkest region in the image) will not be increased uniformly when the diaphragm is enlarged; instead, more information on the shadow region is recorded as contrast intensifies. In the case of image data, however, since brightness information on the shadow region has already been quantized and compressed, xcex3 correction would not increase the brightness information on the shadow region. In other words, the shadow region becomes brighter uniformly to an excessive level.
In a video camera, the value of the quantity of light in the backlight portion is always near xe2x80x9c0xe2x80x9d. In an image data input from a color photographic film through a scanner, the value of the light quantity in the backlight portion does not always represent the pixel value near xe2x80x9c0xe2x80x9d. For a photography printed with the backlight portion slightly brighter, the backlight portion is in the vicinity of, for example, the pixel value xe2x80x9c30xe2x80x9d. With reference to FIG. 16, an input backlight unit 1604 corresponds to the backlight portion. Through the xcex3 correction for the image data, the brightness value of the backlight portion is converted into an output backlight portion 1605. As illustrated in FIG. 16, the range of the output backlight portion 1605 in this case is narrower than the range of the input backlight portion 1604. In other words, although the backlight portion in the image becomes brighter, the range of the brightness becomes narrower, thereby deteriorating the contrast condition and consequently the image quality.
The conventional backlight correction technique for digital image data, although it simulates the diaphragm control by a camera through the xcex3 correction for the data, is actually incapable of performing a good backlight correction in the correction of the digital data.
One objective of the present invention is to provide an image processing system capable of easily executing such complicated correction operations as correction according to the light source colors and correction of a snowy scenery.
Another objective of the present invention is to provide an image processing system capable of reducing time required for operation and consequently reducing manhours through increased efficiency as well as allowing any operator to obtain desired correction results, regardless of the level of skill and experience.
One more objective of the present invention is to provide an image processing system capable of correcting backlight without decreasing the contrast by giving the characteristic of correction to a greater brightness the medial pixels brighter than the shadow region.
According to the first aspect of the invention, an image processing system which conducts necessary correction processing on digital image data, comprises
correction pattern storing means for storing correction patterns set corresponding to kinds of color fogs,
correction pattern selecting means for selecting the correction pattern suitable for correction of input image among the correction patterns stored in the correction pattern storing means through the operation of an operator,
correction amount storing means for storing a correction amount indicative of an appropriate degree of correction on color fog,
tone curve generating means for generating a tone curve of RGB based on the correction pattern selected by the correction pattern selecting means and the correction amount stored in the correction amount storing means, and
LUT converting means for performing table conversion on image data of the input image based on the tone curve generated by the tone curve generating means to conduct color conversion processing of all the pixels of the image data.
The image processing system may further comprise correction amount specifying means for specifying a correction amount to be stored in the correction amount storing means according to each correction processing by the operation of an operator.
The tone curve generating means may comprise
highlight control amount calculating means for calculating a control amount of a highlight region in each component of RGB based on the correction pattern and the correction amount,
half tone control amount calculating means for calculating a control amount of a half tone region in each component of RGB based on the correction pattern and the correction amount,
shadow control amount calculating means for calculating a control amount of a shadow region in each component of RGB based on the correction pattern and the correction amount,
tone curve calculating means for calculating the tone curve of each component of RGB based on the three kinds of control amounts calculated by the three kinds of control amount calculating means,
a straight line generating module for use in the tone curve calculating processing by the tone curve calculating means to conduct straight line interpolation between a control point in a high brightness region and a control point in a low brightness region, and
a curve generating means for use in the tone curve calculating processing by the tone curve calculating means to conduct curve interpolation for smoothly interpolating between a control point in a half tone region and starting points on the high brightness side and the low brightness side.
According to the second aspect of the invention, digital image data correcting method in an image processing system which conducts necessary correction processing on digital image data, comprises the steps of
among correction patterns set corresponding to kinds of color fogs, selecting the correction pattern suitable for correction of input image by the operation of an operator,
obtaining a correction amount indicative of an appropriate degree of correction on color fog,
generating a tone curve of RGB based on the correction pattern selected at the correction pattern selecting step and the correction amount obtained at the correction amount obtaining step, and
performing table conversion on image data of the input image according to the tone curve generated at the tone curve generating step to conduct color conversion of all pixels of the image data.
The tone curve generating step comprises the steps of
calculating a control amount of a highlight region in each component of RGB based on the correction pattern and the correction amount,
calculating a control amount of a half tone region in each component of RGB based on the correction pattern and the correction amount,
calculating a control amount of a shadow region in each component of RGB based on the correction pattern and the correction amount, and
calculating the tone curve of each component of RGB based on the three kinds of control amounts calculated at each the steps, wherein
at the time of calculating a tone curve, the tone curve calculating step performing
a straight line generating module for conducting straight line interpolation between a control point in a high brightness region and a control point in a low brightness region, and
a curve generating module for conducting curve interpolation for smoothly interpolating between a control point in a half tone region and starting points on the high brightness side and on the low brightness side.
According to the third aspect of the invention, a computer readable memory which stores an image processing program for controlling an image processing system which conducts necessary correction processing on digital image data, the image processing program comprises the steps of
among correction patterns set corresponding to kinds of color fogs, selecting the correction pattern suitable for correction of input image by the operation of an operator,
obtaining a correction amount indicative of an appropriate degree of correction on color fog,
generating a tone curve of RGB based on the correction pattern selected at the correction pattern selecting step and the correction amount obtained at the correction amount obtaining step, and
performing table conversion on image data of the input image according to the tone curve generated at the tone curve generating step to conduct color conversion of all pixels of the image data
According to the fourth aspect of the invention, an image processing system which conducts necessary correction processing on digital image data, comprises
shadow region brightness value extracting means for extracting a shadow region brightness value which is a brightness value of a low brightness pixel from image data of input image,
backlight correction amount storing means for storing a backlight correction amount indicative of an appropriate degree of correction on backlight input image,
backlight correction table generating means for generating a backlight correction table characterized by, based on the shadow region brightness value extracted by the shadow region brightness value extracting means and the backlight correction amount stored in the backlight correction amount storing means, not correcting a pixel whose brightness value falls in the range between xe2x80x9c0xe2x80x9d and the shadow region brightness value and correcting a pixel whose brightness value is higher than the shadow region brightness value so as to increase the brightness value, and
LUT converting means for performing table conversion on image data of the input image based on the backlight correction table generated by the backlight correction table generating means to conduct color conversion processing of all pixels of the image data.
The image processing system may further comprise RGB brightness value converting means for calculating, from an RGB value of a pixel, a brightness value of the pixel, wherein the shadow region brightness value extracting means causes the RGB brightness value converting means to extract a shadow region brightness value of a desired low brightness pixel.
The backlight correction table generating means may comprise
parameter calculating means for calculating a parameter for use in correction based on the backlight correction amount stored in the backlight correction amount storing means,
first conversion table generating means for generating a conversion table by using the parameter calculated by the parameter calculating means, and
second conversion table generating means for generating the backlight correction table based on the conversion table generated by the first conversion table generating means and the shadow region brightness value extracted by the shadow region brightness value extracting means,
the parameter calculating means calculating spline control point coordinates based on the backlight correction amount, and
the first conversion table generating means calculating a spline curve based on the spline control point coordinates.
The backlight correction table generating means may comprise
parameter calculating means for calculating a parameter for use in correction based on the backlight correction amount stored in the backlight correction amount storing means,
first conversion table generating means for generating a conversion table by using the parameter calculated by the parameter calculating means, and
second conversion table generating means for generating the backlight correction table based on the conversion table generated by the first conversion table generating means and the shadow region brightness value extracted by the shadow region brightness value extracting means,
the second conversion table generating means may include
backlight correction characteristics calculating means for calculating, with reference to a brightness value of a pixel in image data of input image, the shadow region brightness value, and the conversion table generated by the first conversion table generating means, a brightness value of the pixel obtained after backlight correction, and
brightness value comparing means for comparing a brightness value of each pixel in image data of input image with the shadow region brightness value, and when the brightness value of the target pixel is lower than the shadow region brightness value, taking the brightness value of the target pixel as a brightness value to be obtained after the correction and when the brightness value of the target pixel is higher than the shadow region brightness value, activating the backlight correction characteristics calculating means to calculate a brightness value to be obtained after the correction.
According to the fifth aspect of the invention, a digital image data correcting method in an image processing system which conducts necessary correction processing on digital image data, comprises the steps of
extracting a shadow region brightness value which is a brightness value of a low brightness pixel from image data of input image,
obtaining a backlight correction amount indicative of an appropriate degree of correction on backlight input image,
generating a backlight correction table characterized by, based on the shadow region brightness value extracted at the shadow region brightness value extracting step and the backlight correction amount obtained at the backlight correction amount obtaining step, not correcting a pixel whose brightness value falls in the range between xe2x80x9c0xe2x80x9d and the shadow region brightness value and correcting a pixel whose brightness value is higher than the shadow region brightness value so as to increase the brightness value, and
performing table conversion on image data of the input image based on the backlight correction table generated at the backlight correction table generating step to conduct color conversion processing of all the pixels of the image data.
The backlight correction table generating step comprises the steps of
calculating a parameter for use in correction based on the backlight correction amount stored in the backlight correction amount storing means,
generating a conversion table by using the parameter calculated at the parameter calculating step, and
generating the backlight correction table based on the conversion table generated at the conversion table generating step and the shadow region brightness value extracted by the shadow region brightness value extracting means,
the step of generating a backlight correction table based on the conversion table include the steps of
comparing a brightness value of each pixel in image data of input image with the shadow region brightness value,
determining a brightness value of the target pixel as a brightness value to be obtained after the correction when the brightness value of the target pixel is lower than the shadow region brightness value, and
calculating a brightness value of a target pixel to be obtained after the backlight correction with reference to the brightness value of the target pixel, the shadow region brightness value, and the conversion table generated at the conversion table generating step when the brightness value of the target pixel is higher than the shadow region brightness value.
According to the sixth aspect of the invention, a computer readable memory which stores an image processing program for controlling an image processing system which conducts necessary correction processing on digital image data, the image processing program comprises the steps of
extracting a shadow region brightness value which is a brightness value of a low brightness pixel from image data of input image,
obtaining a backlight correction amount indicative of an appropriate degree of correction on backlight input image,
generating a backlight correction table characterized by, based on the shadow region brightness value extracted at the shadow region brightness value extracting step and the backlight correction amount obtained at the backlight correction amount obtaining step, not correcting a pixel whose brightness value falls in the range between xe2x80x9c0xe2x80x9d and the shadow region brightness value and correcting a pixel whose brightness value is higher than the shadow region brightness value so as to increase the brightness value, and
performing table conversion on image data of the input image based on the backlight correction table generated at the backlight correction table generating step to conduct color conversion processing of all the pixels of the image data.
Other objects, features and advantages of the present invention will become clear from the detailed description given herebelow.